An Information-based Approach for Guiding Multi-Modal Human-Computer-Interaction

Much work has been done in dialogue mod-eling for spoken and multi-modal human-computer interaction. Problems can arise in situations that do not correspond to the dialogue model. For this reason, we propose information-centered dialogue processing in which the actions to be taken by the dialogue system are determined as a function of the information available in the discourse, the database and the domain model. In order to arrive at fully speciied representations of the intended actions, the speciicity of the representations is increased by uniication, integrating information from multi-modal input, database access and domain knowledge. Our approach diiers from other state-of-the-art systems in that it does not rely on explicit dialogue models. Instead, we show how partial and under-speciied representations of the situation can be used in a spoken dialogue system to generate clariication questions and to guide the user to arrive at his or her communicative goal. We show furthermore how probabilistic information can be used to disambiguate without clariication questions. Evaluation results and dialogue examples demonstrate the exibility and naturalness of our approach.